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ABSTRACT 



As part of a study of the inclusion of English language 
learners (ELLs) in large-scale assessments, this study investigated the 
potential role of student preference in the use of accommodations with 
large-scale assessments and the effects of two test accommodations, providing 
extra assessment time and reading test items aloud, on the standardized test 
performance of 106 ELLs in seventh-grade social studies classrooms. In 
addition, the study investigated the relationships among the accommodations 
students preferred and these background variables: (1) length of time in the 

United States; (2) amount of education prior to arrival in the United States; 
and (3) English language proficiency. Students completed a background 
questionnaire and a standardized test in social studies without 
accommodations. They were then asked to identify which of the two 
accommodations they would prefer. A parallel form of the test was 
administered with accommodations. One third of the students received the 
accommodation they preferred, while a third received the accommodation not of 
their preferences, and the other third received one of the accommodations at 
random. Data analysis found that students did not improve their performance 
significantly with either accommodation, even when given their preferred 
accommodation. No significant relationships were found between accommodation 
preference and various background variables. An appendix contains the study 
questionnaire. (Contains 12 tables and 21 references.) (SLD) 
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THE IMPACT OF PREFERENCE FOR ACCOMMODATIONS: 
THE PERFORMANCE OF ENGLISH LANGUAGE LEARNERS ON 
LARGE-SCALE ACADEMIC ACHIEVEMENT TESTS 1 



Martha Castellon-Wellington 
CRESST/University of California, Los Angeles 

Abstract 

In an effort to include English language learners (ELLs) in large-scale, content- 
knowledge assessments, some educators have begun to use accommodations, defined as 
modifications to a test or to the manner in which a test is administered. To date, 
however, there is no conclusive evidence that accommodations aid students in 
demonstrating their content knowledge on standardized tests. 

This study investigates the potential role of student preference in the use of 
accommodations with large-scale assessments and the effects of two test 
accommodations — providing extra assessment time and reading test items aloud — on the 
standardized test performance of ELLs in seventh-grade social studies classrooms. 
Additionally, the study investigates the relationships between the accommodations 
students prefer and the following background variables: their length of time in the 
United States, amount of education prior to arrival in the United States, and English 
language proficiency. 

Students completed a background questionnaire and a standardized test in social 
studies without accommodations. Students then were asked to identify which of two 
accommodations they preferred. A parallel form of the standardized test was 
administered with accommodations. One third of the students received the 
accommodation of their preference, a third received the accommodation not of their 
preference, and a third received one of the two accommodations at random. 

Data analysis found that students did not significantly improve their performance on 
the accommodated test with either accommodation, even when given their preferred 
accommodation. Data analysis also found no significant relationships between 
accommodation preference and various background variables. 



Standardized tests traditionally have been a controversial issue among 
educators, policymakers, and parents alike. Questions concerning their usefulness 
and appropriateness are continually debated. Do the standardized tests that are 



1 The work reported here originally appeared in: Castellon, M. (1999). The impact of preference for 
accommodations: The performance of English language learners on large-scale academic achievement tests. 
Unpublished master's thesis. University of California, Los Angeles. This report is slightly modified 
from the original. 



employed in school districts today accurately measure what students are being 
taught in school? Are students required to respond to the items included in tests in a 
manner similar to that which is required of them in the classroom? Questions such 
as these and many more are fodder for discussion for teachers, parents, and others 
who are concerned that students are given a fair shake in the assessment game. 

The current debate over how to provide students with limited English 
proficiency in grades kindergarten through 12 with equal access to the curriculum 
complicates the issue of assessment even further. Not only are educators concerned 
with making sure ELLs access the same rich curriculum as their native-English- 
speaking counterparts, educators also are concerned with the inclusion of these 
students in large-scale assessments. According to Butler and Stevens (1997): 

Students who speak English as a second language are often systematically excluded from 

large-scale assessments, in spite of Title I provisions of the Improving America's Schools 

Act which require that ELLs be included in assessments to the extent practicable, (p. 1) 

The work of Cummins and Collier helps to shed light on the reasons why 
students with limited English proficiency are continually excluded from large-scale 
assessments. According to Cummins (1979, 1980), most ELLs take between one and 
three years to develop the basic interpersonal communication skills (BICS) that 
allow them to communicate in English at a very superficial level with peers and 
teachers. The time it takes ELLs to acquire the cognitive academic language 
proficiency (CALP) that is necessary for them to actively participate in their 
classroom learning, however, is between five and seven years (Collier, 1987; 
Cummins, 1981). 

These findings lead researchers to ask a variety of questions that address issues 
of equity in large-scale assessments. If students for whom English is a second 
language are still acquiring knowledge of academic vocabulary and sentence 
structure in addition to academic language functions such as comparing, 
contrasting, analyzing, and evaluating, are their scores on content-based 
assessments as meaningful as the scores of students who have spoken English all of 
their lives? If test scores are used to make comparisons with regard to the 
performance of native and non-native English speakers, are those comparisons fair? 
Furthermore, to what extent do ELLs' scores on content-based assessments reflect 
their knowledge of the content, and to what extent do they reflect their difficulties in 
interpreting the language used in the test stimuli? 



While the rationale for excluding ELLs from assessments designed for students 
in mainstream classrooms seems reasonable, a consequence of such exclusion is that 
they are not held to the same high standards of achievement as their native-English- 
speaking peers. That's because no standardized data exist to facilitate the tracking of 
these students' progress nor the identification of their strengths and weaknesses. 

To date there have been few options for alternative methods of assessment for 
these students. For students in elementary bilingual classrooms, standardized tests 
in academic content areas such as language arts and math are administered in the 
students' native language when available. In an effort to include ELLs in large-scale 
assessments where testing in the native language is not possible or not appropriate, 
educators in some states have begun to use accommodations or test modifications as 
a means of facilitating the assessment of ELLs' content knowledge. 

Butler and Stevens (1997) define accommodations as "support provided 
students for a given testing event either through modification of the test itself or 
through modification of the testing procedure" (p. 5). Types of test modifications 
that currently are being investigated include simplifying the English language used 
in test items (Abedi, Lord, & Plummer, 1997; Aguirre-Munoz, 2000), providing 
translations of the test questions in the students' native language (Abedi et al., 1997; 
Aguirre-Munoz, 2000), the reading aloud of test items by the test administrator, and 
allowing students to use an English glossary during the test (Abedi, Hofstetter, 
Baker, & Lord, 1998). Test modifications also may include the addition of visual 
supports (Butler & Stevens, 1997), although no studies have investigated this 
strategy to date. Modifications in testing procedure that also are under investigation 
include providing extra assessment time (Abedi, Hofstetter, Baker, & Lord, 1998) 
and the use of oral directions in the students' native language. 

This study investigates two accommodations: providing extra assessment time 
and reading the test items and directions aloud. The rationale for selecting extra 
assessment time as an accommodation is based on the premise that if language itself 
poses a problem for ELLs, students under normal testing conditions may not be able 
to carefully consider all of the items on the test. Allowing them more time to 
complete the test may lead to their being able to respond accurately to more items. 
The rationale for investigating the read-aloud accommodation is that some students 
may be more prone to respond to both visual and oral stimuli than they would be to 
visual stimuli alone. For example, if a student does not understand one or more of 
the words on a test item, having the test administrator read the words aloud in the 



context of the entire question, using proper intonation, might help the student better 
understand the question and hence respond appropriately. The fact that both types 
of accommodations could be implemented without additional testing materials and 
without modifications of existing testing materials also played a part in their 
selection for this study. 

As will be noted in the review of the literature, the number of studies that have 
been conducted on the use of accommodations in large-scale assessments are few 
and have been at best preliminary in their findings. While some have addressed the 
question of which accommodations are most appropriate for particular subgroups of 
students, none as of this writing have addressed the issue of considering students' 
preferences as a means of selecting an appropriate accommodation. This study aims 
to investigate whether students are able to select the accommodation that will allow 
them to better demonstrate the content knowledge they have acquired. 

Research Questions 

The research questions for this study are as follows: 

• Do ELLs perform significantly better on a test of content knowledge when 
given either one of the accommodations used in this study? 

• Do ELLs perform significantly better on a test of content knowledge when 
given their choice of accommodation with the assessment? 

• Is there a relationship between the type of accommodation students prefer 
and the following background variables: the amount of time students have 
lived in the United States, the amount of formal education students received 
prior to their arrival in the United States, and students' English language 
proficiency? 

The first question addresses whether either one of the accommodations affects 
student performance, whereas the second question focuses on the effect of providing 
students with their preferred accommodation. The third question can be viewed as 
an outgrowth of question number two. If, for example, this study revealed that 
receiving their preferred accommodation did in fact make a difference in students' 
test performance, then it would be important to investigate whether there are 
similarities among the types of students who select each accommodation. 
Identifying such patterns might aid test administrators in properly assigning 
accommodations in the future. 



Limitations of the Study 

The limitations of the study fall into two categories: threats to the study's 
external validity and threats to the study's internal validity. Potential threats to the 
study's external validity address the ability to generalize the findings. It is important 
to note that the findings from this study might not generalize to other students and 
other settings because it is unknown to what extent the sample used in this study is 
representative of ELLs in other situations. A description of the characteristics of the 
test takers in this study is included in the Methodology section. 

A second potential threat to the study's external validity is the fact that the 
results might not generalize to other standardized achievement tests, as each test is 
different, and may not assess the same constructs in the same way as the 
standardized tests used in this study. Finally, a third potential threat to the study's 
external validity is the possibility that the results might not generalize to other types 
of accommodations. 

Threats to the internal validity of the study include the extent to which one can 
attribute the study's outcome to the treatment, which, in this case, is a combination 
of preference and accommodation. It is important to note that the sample of students 
selected for this study was not composed of students who were randomly selected 
from the entire population of seventh grade ELLs in the Burbank Unified School 
District. Rather, the students in the sample were selected because of their 
membership in previously formed classroom groups. As a result, the six classroom 
groups participating in the study were not completely homogenous in composition. 
One classroom in particular contained a large percentage of students with relatively 
low levels of English proficiency, while other classrooms contained a relatively small 
percentage of students with low levels of English proficiency. These pre-existing 
differences could account in part for any cross-classroom discrepancies in the 
performance of students on the accommodated versions of the test. This quasi- 
experimental research design, then, makes it impossible to control for pre-existing 
differences in the groups selected, and consequently poses a potential threat to the 
internal validity of the study. 

Assumptions 

In considering the procedures described in the Methodology section of this 
report, the following assumptions were made: 



• Students took the tests seriously and attempted to do their best on each one. 

• Students were not coached by the teacher, test administrator, or other 
individual during the test; all work was their own. 

• Standard test administration procedures were followed. 

• The statistical assumptions underlying multiple regression analysis were 
not violated. (This will be addressed further in the results section). 

Significance of the Study 

The study is significant in several respects. First, it is significant in that it will 
help advance our present understanding of the effects of accommodations with 
ELLs. Such understanding is critical in determining whether accommodations 
suggest a practical solution to the problem of assessing the content knowledge of 
ELLs in English. This study also is significant because it will help to elucidate the 
role that student preference may play in the assignment of accommodations in the 
future. 

Because so little research has been conducted in test accommodations for ELLs 
to date, this study is considered exploratory. In addition to helping to answer the 
research questions cited above, this study is meant to provide practical experience 
that will help researchers design future accommodations studies more effectively. 

Review of the Literature 

Although practitioners and researchers have been in agreement on the 
limitations of using standardized tests for assessing achievement among ELLs 
(Ascher, 1991; Olmedo, 1981; Zirkel, 1972), to date little has been published 
regarding the types of accommodations that may be used with students from 
diverse cultural and linguistic backgrounds (August & Hakuta, 1997; Thurlow, Liu, 
Erickson, Spicuzza, & El Sawaf, 1996). Consequently, no solid research base exists to 
support the use of accommodations on large-scale assessments. In reviewing the 
extant literature, I will provide a rationale for the study of accommodations as an 
alternative to traditional assessments for ELLs, discuss previous research on 
preference as it relates to writing assessments, identify the issues to be considered 
and the steps that need to be taken prior to the implementation of accommodations, 
and finally, cite studies that suggest the implications of their use. 



Accommodations: A Rationale 

The rationale for the use of accommodations on large-scale assessments is 
based on a growing body of evidence that ELLs' knowledge of content frequently 
goes underestimated using current methods of assessment (LaCelle-Peterson & 
Rivera, 1994). In an effort to examine the role that language plays in the performance 
of ELLs on content-based assessments, Abedi et al. (1997) conducted a study using 
data from the mathematics section of the National Assessment of Educational 
Progress (NAEP) for eighth grade. In this study, the researchers found that subjects 
who predominantly spoke a language other than English in the home had lower 
overall math proficiency scores than students who reported speaking only English at 
home. Furthermore, the difference in scores between the two groups was found to 
be greater for those test items that were identified by the researchers as linguistically 
complex. 

Other findings further reveal language to be a confounding factor on the 
mathematics performance of ELLs. Students in the same NAEP study were grouped 
according to whether they reported always, sometimes, or never speaking a 
language other than English in the home. Upon examination of the proportion of 
omitted or not-reached items by students in these groups, researchers found that in 
nearly every case, students who always spoke a language other than English in the 
home had much higher percentages of omitted/ not-reached items than the students 
who spoke only English at home. 

Although these studies were conducted in only one content area, mathematics, 
their findings are important because they suggest what educators long have 
considered to be true: language may impede the performance of ELLs on content 
assessments. Such findings have prompted researchers to call on states and school 
districts to responsibly report on the educational progress of ELLs in terms of their 
developing English proficiency and content knowledge (LaCelle-Peterson & Rivera, 
1994). 

Preference in Writing Prompts 

Although the issue of accommodation preference has not been investigated 
with respect to large-scale standardized assessments for ELLs, student preference of 
writing prompts has long been used in writing assessments and is well documented. 
The rationale for allowing students to select a writing prompt is remarkably similar 
to the rationale for allowing students to select their own accommodation. According 



to Polio and Glew (1996), "the primary reason for offering students a choice of 
prompts is the belief that students should be allowed to choose a prompt that will 
enable them to display their best writing." Indeed university students themselves 
lend credence to such reasoning as 24 of the 26 English-as-a-Second-Language (ESL) 
students tested felt that they chose the writing prompt that best allowed them to 
demonstrate their essay-writing ability (Polio & Glew, 1996). 

Interestingly, a study of high school students who took Advanced Placement 
tests in U.S. and European history found that examinees generally made the correct 
choice when selecting the writing topic on which they could get a higher score. In 
their 1997 study, Bridgeman, Morgan, and Wang found that 52% of students taking 
the Advanced Placement examination in U.S. history received higher scores on their 
preferred topic. The same was found to be true for 58% of the students who took the 
European history test. While these studies do not involve the same type of students 
as those investigated in the present study, their findings indicate that students are 
often able to identify the mechanisms that allow them to perform at their best. 

Guidelines and Considerations 

As a way of providing initial guidance to researchers and educators, Butler and 
Stevens (1997) offer a comprehensive look at the issues that must be addressed in 
considering the use of accommodations with ELLs on large-scale assessments. 
Among the questions they consider are (a) How do educators decide which 
accommodations are suitable for which students? (b) To what extent do 
accommodations impact student performance? and (c) What are the effects of the 
accommodations on the validity of the assessments with which they are used? 
Rather than suggest that accommodations are the answer to the problem of 
assessment for ELLs, the authors stress the need to determine through solid research 
the effectiveness of various types of accommodations in a range of assessments, the 
need to identify subgroups of students who may benefit from the use of 
accommodations, and the need to establish guidelines for their use. 

Another consideration mentioned in the work of Thurlow et al. (1996) — one 
that is not addressed in this study — is that of determining which types of 
accommodations allow ELLs to demonstrate what they know and which types of 
accommodations give them an unfair advantage over monolingual students. Future 
studies must clearly address this issue if states are to adhere to the principle of fair 
and equitable assessment for all students. 



A final mention should be made in regard to the practicality of administering 
accommodations on large-scale assessments. According to Abedi, Lord, and 
Hofstetter (1998), "more attention should be given to the feasibility of administering 
different forms of accommodations for LEP [limited English proficient] students. If 
the most effective form of accommodation is not practical or logistically possible, it 
may not be useful" (p. ix). 

Identifying Background Variables 

According to Butler and Stevens (1997), determining which background 
variables are most indicative of student performance may aid assessment specialists 
in deciding which groups of students would benefit from various accommodations. 
Researchers have identified the following background variables for further 
investigation: English language proficiency (Abedi, Hofstetter, Baker, & Lord, 1998; 
Abedi, Lord, & Hofstetter, 1998; Butler & Stevens, 1997), amount of formal schooling 
prior to arrival in the United States (Butler & Stevens, 1997), and length of time in 
the United States (Abedi, Lord, & Hofstetter, 1998; Butler & Stevens, 1997). All three 
of these variables are investigated in the present study. 

Studies on Linguistic Modifications and Native Language Assessment 

After finding that language did in fact play a role in ELLs' NAEP mathematics 
performance, Abedi et al. (1997) modified 20 original NAEP math items to reduce 
their linguistic complexity’. Two test booklets were prepared, one containing the 20 
original linguistically complex items, and one containing the modified versions of 
the original items. After both booklets were administered to a sampling of 1,031 
native- and non-native-English-speaking eighth graders, ELLs showed no 
statistically significant gains on the modified version. Their scores overall, however, 
were found to be significantly lower than those of native-English-speaking students. 

In a separate NAEP study, Abedi, Lord, and Hofstetter (1998) administered test 
items to three groups of students at random. One group received test items in 
English as they appeared on the original assessment. A second group received 
linguistically modified test items in English, and a third group received the test 
items in Spanish. Subsequent analyses revealed that students scored best on the 
linguistically modified test items in English (where all students demonstrated 
marked improvement), and worst on the test items in Spanish. 



Studies on the Use of Glossaries and Extra Assessment Time 

Common among today's practitioners is the belief that bilingual students 
process information more slowly in their less familiar language, which in turn 
accounts for their slower test-taking speed (Ascher, 1991). LaCelle-Peterson and 
Rivera (1994), however, attribute the need for additional testing time to differences 
in the rate at which students learn a second language. The issue of how much 
assessment time to allow ELLs prompted Abedi, Hofstetter, Baker, and Lord (1998) 
to conduct a study in which students were administered 35 NAEP mathematics 
items with one of the following accommodations: extra assessment time, the use of a 
glossary for non-mathematical terms, and the use of a glossary plus extra 
assessment time. Although students benefited from having extra assessment time, 
ELLs were found to benefit more from the double accommodation of a glossary plus 
extra time. Students showed no significant improvement when allowed to use a 
glossary without extra time, possibly due to the time that elapsed while consulting 
the glossary. 

Implications 

Results from the Abedi, Lord, and Hofstetter study (1998) indicate that all 
students would benefit from more clearly worded math problems. Modifying the 
language of NAEP test items in mathematics does not always yield improved test 
scores for ELLs, however, as indicated by Abedi et al. (1997) above. More research is 
necessary to determine when and for whom modifying the language of test items is 
beneficial. 

Regarding the use of translation as an accommodation strategy, poor 
performance on the Spanish version in the second NAEP study suggests that 
students tend to perform best when the language of the test matches the language of 
mathematics instruction (Abedi, Lord, & Hofstetter, 1998). 

In terms of the glossary-plus-extra-time accommodation, its success prompts 
questions as to whether the use of combined accommodations is more effective than 
their individual use. Finally, more studies are needed to test the efficacy of these 
accommodations in other content areas. 



Methodology 

The study's design was quasi-experimental and included a pre-test and a post- 
test, multiple treatments, and multiple comparison groups. 

Setting 

The study took place in Burbank, California, a quiet, mostly middle-class, 
predominantly white suburb of Los Angeles. Burbank's immigrant population is 
composed primarily of Latinos and Armenians; however, other language groups 
also are represented. 

Of the 14,350 students enrolled in the school district in 1997, 6,713 (46.8%) 
spoke languages other than English at home. Table 1 shows the breakdown of the 
other languages spoken. 

Of the 6,713 students who spoke languages other than English at home in 1997, 
3,277 (or 22.8% of the total student population in Burbank Unified School District) 
were designated limited English proficient (LEP). 

Students in 6 seventh-grade social studies classrooms from the Burbank 
Unified School District participated in this study. All were selected to participate in 
the study on the basis of their "sheltered" status by the district's coordinator of 
student and program assessment and evaluation. Sheltered classrooms are 
composed of ELLs whose low English language proficiency does not allow them to 
be placed in regular mainstream English classrooms. The content in sheltered 
classrooms is delivered by the teacher using special techniques designed to make the 
content more accessible to students whose English skills are still developing. 

Table 1 



Other Languages Spoken in the Burbank Unified School District 



# of students who 
speak the language 


% of total district 
population 


Spanish 


3,972 


27.7 


Armenian 


1,241 


8.7 


Korean 


375 


2.6 


Filipino (Tagalog) 


273 


1.9 


Arabic 


214 


1.5 


Vietnamese 


92 


.6 


Other 


546 


3.8 


Total 


6,713 


46.8 



Several differences among participating classrooms should be noted. The six 
participating classrooms represented three different Burbank schools. The schools 
generally reflected the demographics of the entire school district. Table 2 indicates 
the school with which each classroom was associated as well as the various class 
configurations represented in the research sample. 

Classrooms A and B consisted exclusively of ELLs who received sheltered 
instruction in social studies. Classroom C, however, was composed of a mix of 
sheltered and mainstream English-only (EO) students. All non-LEP students from 
this classroom were excluded from all research activities at the request of the school. 

Classrooms D, E, and F all came from School 3, and classrooms D and F were 
also composed of a mix of sheltered and mainstream English-only students. 
Although some students in these classrooms were not ELLs, all students 
participated in every survey and test that was administered. Classroom E was a 
special case. This was a designated English language development (ELD) class 
because its students were, on the whole, newer to the United States than students 
from other classrooms. Consequently, their English skills were on average lower 
than those of ELLs in the other classrooms. Despite the students' ELD status, the 
content in this classroom also was delivered using sheltered techniques. 

All three of the schools participating in this study offered support for their ELL 
populations with sheltered and ELD classrooms. The resources available to ELLs at 
each school site were therefore comparable. 

Table 2 



Participating Schools and Classrooms 





School 1 


School 2 


School 3 


Classroom A 


Sheltered 






Classroom B 




Sheltered 




Classroom C 




Sheltered & EO 3 




Classroom D 






Sheltered & EO 


Classroom E 






ELD 


Classroom F 






Sheltered & EO 



Note. EO = English only; ELD = English language development. 

a All non-LEP students from this classroom were excluded from all research activities 
at the request of the school. 












Participants 

The sample consisted of 106 seventh-grade ELLs in six classes as illustrated in 
Table 3. 

Table 4 displays the countries of origin of the 48 students in the sample who 
were bom outside the United States. 



Table 3 

Number of ELLs From Each Classroom 



Classroom 


# of ELLs 


Classroom A 


18 


Classroom B 


27 


Classroom C 


12 


Classroom D 


13 


Classroom E 


20 


Classroom F 


16 


Total 


106 



Table 4 

Countries of Origin 


Countries 


Number of students 


Armenia 


14 


Mexico 


13 


Iran 


4 


Philippines 


4 


Syria 


3 


Germany 


2 


Ecuador 


1 


Egypt 


1 


El Salvador 


1 


India 


1 


Nicaragua 


1 


Pakistan 


1 


Russia 


1 


Thailand 


1 


Total 


48 



ERiC 



13 



Of the 106 students in the sample, 101 reported speaking at least one language 
other than English at home (Table 5). Furthermore, 50 students reported speaking 
two or more languages (English included in some cases) at home. 

Of the 48 students who were bom outside of the United States, 10 had lived 
here less than one year, 10 had lived here between one and three years, and 28 
reported having lived in the United States for more than three years. Further, 44 out 
of the 106 students in this sample had attended school outside the United States. 
Table 6 shows the number of years these students attended school in other countries. 

Table 5 



Languages Spoken in the Home 



Languages 


Number of students 


Spanish & English 


36 


Spanish 


29 


Armenian 


11 


Armenian & English 


9 


Arabic 


4 


Tagalog 


4 


Armenian & Russian 


1 


Armenian, Persian & German 


1 


Armenian, Persian, German & English 


1 


Armenian, Russian & English 


1 


Persian & English 


1 


Punjabi 


1 


Thai 


1 


Urdu 


1 


Total 


101 



Table 6 

Number of Years of School 


in Other Country 


Years 


Number of students 


One year or less 


12 


2 years 


9 


3 years 


5 


4 years 


2 


5 years 


3 


6 years or more 


13 


Total 


44 



Instruments 



A student background questionnaire developed by the National Center for 
Research on Evaluation, Standards, and Student Testing (CRESST) was administered 
in order to gather information regarding students' first language, country of origin, 
the number of years they had lived in the United States, the amount of formal 
schooling they had received in their home country, etc. (see Appendix A). Part of the 
data from this questionnaire was used in answering Research Question 2. 

Permission was granted by The Riverside Publishing Company to use two 
parallel forms (K and L) of the Iowa Tests of Basic Skills (ITBS) Social Studies Test 
for 7 th Grade/Level 13 (The Riverside Publishing Company, 1993). The Social 
Studies tests, which consist of 44 multiple-choice items, emphasize the use and 
understanding of concepts, principles, and selected visual materials from the 
following content areas: history, geography, political science, economics, sociology, 
and anthropology. In accordance with the procedures specified in the test 
administration booklet, students had 30 minutes to complete the test. 

Another instrument used in this study consisted of a preference questionnaire, 
also developed by CRESST, in which students were asked to answer the following 
question: "What would help you do better on this test?" Students selected one of 
two available answers: (a) having extra time, and (b) having the test items and 
directions read out loud (see Appendix B). 

Another test was administered in order to assess students' English-reading 
ability. The Language Assessment Scales Reading Component Form 3A (LAS- 
Reading 3A) (Duncan & De Avila, 1990) is a 55-item multiple-choice test consisting 
of five separate subtests: synonyms, fluency, antonyms, mechanics, and reading for 
information. Because the LAS-Reading 3A is not a timed test, students were given as 
much time as necessary in order to complete it. Collecting information on students' 
English-reading ability was necessary in order to answer Research Question 3. 

Procedures 

During the first visit to each classroom, the student background questionnaire 
was administered. Although the questionnaire was piloted and revised before its 
use in this study, some students with lower English proficiency had difficulty 
understanding the wording of several of the questions. When necessary, both the 



researcher and the classroom teacher assisted students with completing the 
background questionnaire. 

On a second visit to each classroom one week later, the social studies test from 
Form L of the ITBS was administered. During the test some students asked 
questions about specific test items, especially with regards to vocabulary. The test 
administrator informed these students that she was unable to help them on the test 
but that they should try their best to select the correct response. Several students 
were unable to complete the entire test either because they did not know the 
answers to the questions or because they ran out of time. The majority of the 
students, however, answered all of the questions on the test. 

Upon finishing the test, each student was given the preference questionnaire. 
Students had little difficulty responding to this question. A few students expressed 
their belief that neither one of the options would allow them to improve their 
performance on a similar test. In these instances, students were asked to indicate this 
belief on the questionnaire, but to select one of the two available choices anyway. 

Two weeks later. Form K of the ITBS social studies test was administered in all 
six participating classrooms using one of the two accommodations under 
investigation. Two classes were given their preferred accommodation, as stated on 
the preference questionnaire; two classes were given the opposite of their preferred 
accommodation; and two classes were randomly assigned one of the two 
accommodations. The criteria used to decide which classrooms would receive each 
treatment were based primarily on the following considerations: (a) classroom 
composition (was the classroom exclusively composed of ELLs, or was the class 
composed of both ELLs and native English speakers as was the case in two of the 
classrooms?) and (b) creating a well-balanced sample (it was important to have 
roughly equal numbers of students in each treatment group). 

Lists of the students who would receive each accommodation were prepared in 
advance of the second ITBS administration. Upon arrival at each classroom, test 
administrators separated students into two groups. Students in the read-aloud 
group were taken to the school library or to another classroom to take the test while 
students in the extra-time group took the test in their own classroom or moved to an 
alternate location. After students were situated in their testing location, test 
administrators read the test directions aloud. Minor modifications to the test 
directions were made so as to maintain consistency with the accommodations being 



used. Students who received extra time were given 50 minutes to complete the test 
(20 minutes extra). This meant that the testing session was conducted in two class 
periods. None of the students in the sample required all 50 minutes to complete the 
test. In fact most students completed the test in less than 40 minutes. Students in the 
read-aloud group were read the entire test out loud and were given up to seven 
minutes to review their responses at the end. Ten to 15 seconds were allowed 
between most test items. On the average, the read-aloud test took about 15 minutes 
longer to administer than the one which allowed extra time. 

During a final visit to each classroom three weeks later, the LAS-Reading 3A 
was administered. Several students in classrooms A, B, C, and E required more than 
one class period to complete the test. 

Table 7 illustrates which classrooms participated in each of the research 
activities. 

Analyses 

Multiple regression analyses were used for Research Questions 1 and 2, as it 
was necessary to control for pre-existing differences among students in the sample. 
Correlational analyses (point biserial and Kendall's Tau were used in answering 
Research Question 3. All statistical analyses were performed using SPSS version 6.1 
for Macintosh (SPSS, 1995). 



Table 7 



Classrooms Participating in Research Activities 





Week 1 


Week 2 


Week 2 


Week 4 


Week 4 


Week 4 


Week 7 


Classroom 


Background 

question- 

naire 


ITBS 

without 

accom. 


Preference 

question- 

naire 


ITBS with 
preferred 
accom. 


ITBS with 
opposite of 
preferred 
accom 


ITBS with 
random 
accom. 


Language 

assessment 

scales 


A 


X 


X 


X 


X 






X 


B 


X 


X 


X 




X 




X 


C 


X 


X 


X 




X 




X 


D 


X 


X 


X 






X 


X 


E 


X 


X 


X 






X 


X 


F 


X 


X 


X 


X 






X 



Note. Accom. = accommodation. 



Results 



This section reports on the reliability coefficients for all three tests used in this 
study. A discussion of the descriptive statistics and statistical assumptions 
underlying multiple regression will follow. Next, the correlations of the variables in 
the regression equation will be reported. Lastly, the results of the regression 
analyses as they relate to all three research questions will be presented. 

Reliability 

In computing the reliability coefficients for all three tests, it was found that the 
reliability coefficient for Form L of the ITBS was .569, and the reliability coefficient 
for Form K was .378. The reliability coefficient for the LAS-Reading 3A was .864. 

Descriptive Statistics 

The descriptive statistics for Forms L and K of the ITBS and the LAS-Reading 
3A are presented in Table 8. 

In Table 8, we see that the mean score on Form L was 12.83 whereas on Form K 
the mean was 15.10. The standard deviation on Form L was 5.03 compared to 4.02 on 
Form K. The mean score on the LAS was 37.49 with a standard deviation of 7.97. 

Statistical Assumptions 

Since the research questions only address the performance and accommodation - 
preference of ELLs, the data from the native-English-speaking students were 
excluded prior to conducting the analyses. Univariate distributions of all of the test 
scores were examined and all were found to be normal. A review of the bivariate 



Table 8 



Descriptive Statistics for the ITBS Forms L & K and for the LAS-Reading 3A 



Statistic 


ITBS Form L 
Not accommodated 


ITBS Form K 
Accommodated 


LAS 

Reading 3A 


N 


96 


96 


96 


# of Items 


44 


44 


55 


Mean 


12.83 


15.10 


37.49 


Std. Deviation 


5.03 


4.02 


7.97 


Min-Max 


0-28 


6-28 


19-55 


Skewness 


.25 


.49 


-.06 


Kurtosis 


.24 


.67 


.47 



scatterplots and plots of standardized residuals showed that the following 
assumptions underlying linear regression were met: 

• Normality 

• Linearity 

• Equal variance 

Correlations 

The product-moment correlations and significance levels of all of the variables 
used in the regression analyses were calculated and are found in Table 9. "Received 
preference" refers to whether or not a student received his/her preferred 
accommodation on the accommodated test and "Accommodation received" refers to 
the accommodation that the student received on Form K. 

As seen in Table 9, the only significant (p < .05) correlation exists between total 
score on Form L and total score on Form K. Hence, it was expected that the test score 
on Form L (not accommodated) would be a significant predictor of performance on 
Form K (accommodated). 

Regression Analyses 

Multiple regression analyses were utilized in answering Research Questions 1 
and 2 in order to control for pre-existing differences among students in their 
knowledge of the content. Research Question 1 addresses the issue of whether 
students perform significantly better on the ITBS when receiving either one of the 
two accommodations used in this study. Research Question 2 addresses whether 

Table 9 



Correlation Matrix for Variables in the Regression Equation 





Total score 


Total score 


Received 


Accommodation 




Form K 


Form L 


preference 


received 


Total score 
Form K 


1.000 








Total score 


.476 


1.000 






Form L 


.000 








Received 


.007 


.060 


1.000 




preference 


.471 


.276 






Accommodation 


.048 


-.003 


-.137 


1.000 


received 


.319 


.487 


.089 





students perform significantly better on the ITBS when receiving the 
accommodation of their preference. Forced entry was used to ensure that all of the 
variables were in the equation using Total Score on Form K of the ITBS as the 
dependent variable. Table 10 presents the raw regression weights ( B ), their 
associated standard errors (SE B), the standardized regression weights (p), the t- 
statistic (t) and the significance levels (p) for all three variables in the equation, as 
well as the multiple R, R 2 , and adjusted R 2 for the regression equation. 

Research Question 1: Do ELLs perform significantly better on a test of content 
knowledge when given either one of the accommodations used in this study? As 
Table 10 indicates, the regression weight for accommodation was not significant, 
indicating that neither one of the accommodations significantly affected student 
performance on Form K of the ITBS. 

Research Question 2: Do ELLs perform significantly better on a test of content 
knowledge when given their choice of accommodation with the assessment? The 
regression weight for receiving one's preference also was not significant, indicating 
that receiving their preferred accommodation did not significantly affect students' 
performance on Form K. The answer to Research Questions 1 and 2, therefore, is that 
neither of the two accommodations used in this study significantly affected 
students' performance even when students received their preferred accommodation. 

Research Question 3: Is there a relationship between the type of 
accommodation students prefer and the following background variables: the 
amount of time students have lived in the United States, the amount of formal 
education students received prior to their arrival in the United States, and students' 
English language proficiency? 

To investigate this question, it was necessary to examine the correlations 
between the accommodations students preferred and each of the aforementioned 

Table 10 



Results of Regression Analysis 



Variable 


B 


SEB 


P 


t 


P 


Total L 


.383344 


.072609 


.476570 


5.280 


.0000 


RecPref 


-.119308 


.730589 


-.014880 


-.163 


.8706 


Accom 


.381614 


.731354 


.047459 


.522 


.6030 



Note. Multiple R = .48. R 2 = .23. Adjusted R 2 = .20. 



background variables. Point biserial correlations were calculated to ascertain the 
degree of relationship between accommodations preferred and number of years 
students attended schools in other countries. The same correlation was computed in 
investigating the relationship between accommodations preferred and English- 
reading ability as measured by the LAS-Reading 3A. Because the data for number of 
years in the United States were ordinal, it was necessary to use Kendall's tau-c in 
order to find the correlation between accommodation preferred and the number of 
years students have lived in the United States. These results may be found in 
Table 11. 

As can be seen in Table 11, no significant correlations were found between the 
accommodations that students preferred and the three background variables in 
question: the amount of time students have lived in the United States, the amount of 
formal education students received prior to their arrival here, and students' English 
language proficiency. 



Discussion 

In this section, explanations for each of the study's outcomes are provided. The 
following topics are covered: research questions, test reliability coefficients, 
additional findings, and concluding remarks. 

Research Questions 

The first finding to be addressed is the fact that neither of the two 
accommodations used in the study had a significant effect on the accommodated test 
performance of students in this sample. The obvious explanation that students could 



Table 11 

Correlations Between Accommodation Preferred and Selected 
Background Variables 



Accommodation 

preferred Significance 


Amount of time in U.S. 


.158 


(N.S.) 




tau-c 




Years attended school 


.009 


(N.S.) 


in other country 


rpbis 




Language Assessment Scales 


.048 


(N.S.) 


(LAS) 


rpbis 





perform equally well without any accommodations, while perhaps appealing, is 
overly simplistic because of other factors that need to be considered. In considering 
possible explanations for why the accommodations did not make a difference, we 
must first explore the issue of opportunity to learn. Many of the skills that were 
tested on the ITBS may not have been taught to the students in the sample either in 
seventh grade or earlier in their schooling. If students had not had the opportunity 
to learn and develop these skills, then no type of accommodation would have 
allowed students to improve their performance on the accommodated test. Indeed, 
this was a concern voiced by several students while responding to the preference 
questionnaire. 

A second possible explanation for why the accommodations were not found to 
significantly impact student performance has to do with the difficulty of the 
vocabulary included on the ITBS. During both ITBS administrations, several 
students asked test administrators to explain the meaning of various words on the 
test. These remarks suggest that many students in the research sample may not have 
been familiar with the type of academic vocabulary used in the ITBS. If this indeed 
were the case, neither one of the accommodations used in this study would have 
mitigated the difficulty of interpreting vocabulary items. Hence, one would expect 
that no significant improvement would have been noted on the accommodated test. 

A third possible explanation for the lack of improvement of test scores on the 
accommodated ITBS addresses the level of English reading ability of the students in 
the sample. As indicated by mean LAS-Reading 3A scores in all six classrooms, 
students tended to score relatively high on the 55-item test (see Table 8). This could 
indicate that students with relatively high English reading levels as determined by 
the LAS-Reading 3A may not benefit from the use of accommodations. Future data 
analysis at CRESST will investigate whether accommodations in fact benefited 
students with lower English reading levels. 

The fact that the sample size in this study consisted of only 106 students should 
not escape mention as it too may have contributed to the insignificance of gains 
made on the accommodated test. In fact, having a larger sample size may have 
ensured a less homogeneous sample, which in itself may have produced different 
results. As mentioned previously, however, the study was meant to be exploratory. 
Rather than answeiing the question of the usefulness of accommodations 
definitively, the study was meant to shed light on the benefits and drawbacks of the 



research design before conducting more costly accommodations studies on a larger 
scale. 

Finally, one must consider the possibility that the accommodations used in this 
study simply do not work. Future studies may find that other accommodations or 
combinations of accommodations yield very different results when used with a 
similar student population. 

In exploring reasons why receiving a preferred accommodation also had no 
impact on student performance, one must consider the possibility that students may 
have chosen their preferred accommodation poorly. Selecting the accommodation 
that would have best suited them would have required students to be able to assess 
their abilities by asking themselves the following question: Do I have greater 
difficulty finishing the test in the time allowed, or do I have greater difficulty 
reading and understanding the test items by myself? It is impossible to estimate 
what percentage of the students would have had the self-awareness necessary to ask 
themselves these questions. Furthermore, due to time constraints, test 
administrators did not discuss the intricacies of what receiving each of the 
accommodations would entail. Had students known that receiving extra time would 
allow them 20 minutes longer to take the exam, and that having the test read aloud 
to them would require that they follow the pace of the test administrator in 
completing the test, they might have chosen the accommodation that was more 
appropriate to their particular needs. 

The fact that no significant correlations were found between student preference 
for accommodation and all three selected background variables (time in the United 
States, amount of formal education prior to arrival here, and level of English 
language proficiency) indicates that student preference for accommodation was a 
highly personal decision . What also must be considered is the possibility that neither 
one of the accommodations may have appealed to a majority of students. Were the 
sample size larger and less homogeneous in terms of language ability, and were 
there more than two accommodations offered as available choices, we might in fact 
have seen a different pattern emerge. 

Test Reliability Coefficients 

Among the many findings from the analyses were the low reliability 
coefficients of Forms L and K of the ITBS. The alpha reliability coefficients found for 
the two tests, .569 and .378 respectively, were below what is considered acceptable 



for a reliability coefficient (normally .80 and above). These findings are due, at least 
in part, to the small variance in the distribution. The distribution of scores was 
extremely positively skewed. In addition, the technical summary for the ITBS 
indicates that only .9% of the norming sample included LEP students (The Riverside 
Publishing Company, 1994). All of these points taken together raise significant 
questions regarding the appropriateness of test items for this sample of ELLs. 

Additional Findings 

In carrying out the regression analyses, it was discovered that the LAS-Reading 
3 A was a significant predictor of performance on Form K. Table 12 shows the results 
of the regression analysis when the following variables were entered into the 
regression equation: total score on ITBS Form L, total score on LAS-Reading 3A, 
number of years of education prior to arrival in the United States, preference 
received, and time in the United States. Again, total score on Form K is the 
dependent variable. 

As seen in the regression analysis discussed previously, total score on Form L is 
a significant predictor on Form K (p < .05). What was not seen previously, however, 
is that performance on the LAS-Reading 3A is also a significant predictor on Form K 
(p < .01). Given these results, one may hypothesize that the higher the score on the 
LAS-Reading 3A, the higher the score will be on Form K of the ITBS. Additionally, 
one may hypothesize that the lower the score on the LAS-Reading 3A, the more the 
student stands to benefit from the accommodations used in this study. Further 
analyses at CRESST will attempt to determine the nature of the relationship between 
these two variables. 



Table 12 

Results of Regression Analysis with LAS-Reading 3A as Independent Variable 



Variable 


B 


SEB 


P 


T 


P 


Total L 


.249730 


.080399 


.311532 


3.106 


.026 


Total LAS 


.223636 


.051805 


.437661 


4.317 


.0000 


Prior ed 


.022942 


.252075 


.016535 


.091 


. 9277 


Rec pref 


-.301726 


.742770 


-.037174 


-.406 


.6856 


<1 yr in U.S. 


1.512436 


2.445639 


.099314 


.618 


.5380 


1-3 yrs in U.S. 


.598542 


2.066801 


.046130 


.290 


.7728 


>3 yrs in U.S. 


.025266 


.868665 


.002844 


.029 


.9769 



Note. Multiple R = .6122. R 2 = .37487. Adjusted R 2 = .32215. 



Concluding Remarks 

Although not named as a research question, one of the reasons for conducting 
this study was to determine the practicality of administering accommodations 
according to student preference. What has been learned from this study is that 
student choice for accommodation is more complex than originally thought. 
Questions regarding students' ability to properly self-select an accommodation and 
the number of accommodations that should be offered as choices to students are 
extremely difficult to answer. Rather than continuing to investigate the efficacy of 
using student-preferred accommodations, it would be more feasible to first 
determine which single accommodation or combination of accommodations are 
most beneficial to students with a given set of language characteristics. For example, 
in the light of results reported in the Additional Findings section above, 
consideration should be given to examining the effects of the same two 
accommodations on populations of students with lower levels of English 
proficiency. 

Further, since opportunity to learn may have played a part in the outcome of 
this study, additional research might focus on determining the usefulness of 
accommodations on tests that are more closely aligned to the classroom curriculum. 
If future studies found that the same two accommodations used in this study were 
effective when used with assessments that contain only material that students have 
been exposed to previously, researchers would be able to more confidently assert 
that accommodations are beneficial where opportunity to learn is not an issue. 
Alternatively, if future studies found that providing extra assessment time and 
reading the test items aloud did not significantly impact student performance on 
tests aligned to classroom curriculum, researchers might reasonably conclude that 
factors other than opportunity to leam affect the usefulness of the accommodations. 

Finally, researchers must concern themselves with the very issue that test 
accommodations are designed to promote: the fair and equitable assessment of all 
students. Addressing questions regarding the validity of using standardized tests 
(with or without accommodations) with ELLs is as important a question as asking 
whether accommodations provide certain groups of students an unfair advantage 
over others. Such questions must not be overlooked and are indeed essential in 
bringing about positive change in traditional assessment practices and the policies 
that affect them. 
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APPENDIX A 





National Center for Research on Evaluation , Standards , 
and Student Testing (CRESST) at UCLA 



February 1998 



Name: Date: 

Age: Grade: 



Student Background Questionnaire 
Please read all questions carefully before answering them. 

1. Were you bom in the United States? 

Yes No 

If no, where were you bom? 

2. How long have you lived in the United States? Check one only. 

Less than one year 

1 to 3 years 

Over 3 years 

All of my life 

3. How many years gave you attended school in the United States? 

Less than one year 

1 to 3 years 

Over 3 years 

4. What language/ s do you speak at home? 

5. In the United States, have you ever been taught in another language, like Spanish or Armenian? 
(not English) 

Yes No 

If yes, answer these questions: 

(a) How many years were you taught in that language? 

(b) What language was it? 

(c) Put a check (S) next to the classes you had in that language 

Language Arts Science 

Math History /Social Science 

Other 
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6 . 



Are all of the classes that you are taking now taught in English? 





















Yes No 

If no, put a check (S) next to the classes taught in another language. 

Language Arts Science 

Math History /Social Science 

Other 

7. How long have you been taught just in English? 

Less than one year 

1 to 3 years 

Over 3 years 

8. Have you ever gone to school in another country? 

Yes No 

If yes, answer these questions: 

(a) What country or countries did you go to school in? 



(b) How many years did you go to school there? 



(c) What language/ s did your teacher speak? 



9. Have you ever studied English in another country? 

® Yes No 

If yes, how many years did you study English in that country? 

10. Have you ever studied another language after school or on the weekend in the United States? 

Yes No 

If yes, answer these questions: 

(a) What language /s have you studied? 

® (b) How many years have you studied the language/ s? 
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CRESST1.2 

Preference Questionnaire 



Name: 



Teacher: 



What would help you do better on this test? Circle only one. 



a. Having more time 

b. Having the test questions read out loud to me 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 

National Library of Education (NLE) 

Educational Resources Information Center (ERIC) TM032325 




NOTICE 

REPRODUCTION BASIS 



This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 






